View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0001196T99X171.00 SKB EagleSWpublic2021-04-13 08:532021-12-23 16:37
Reporter(ALTech) Wooshin Kang Assigned To(SW) Peter ChenDue Date2021-06-03 09:52
PriorityhighSeveritys2-severeReproducibilityhave not tried
Status closedResolutionfixed 
Summary0001196: [Eagle] New function for karaoke APP.
DescriptionBelow is new requirement of SKB for karaoke MIC.
Please implement it.
 
1. Please keep sending data to amp driver to output from speaker.
2. Copy data for speaker from step1 and send it to audio record class.
3. Android Audio Record Class get data using "MediaRecorder.AudioSource MIC type=VOICE_COMMUNICATION"
4. Some Apk will use it. (example, Karaoke app uses it to get score)
 
Thanks.
TagsNo tags attached.
Attach Tags

Users monitoring this issue

User List (ALTech) Younkwang Jung , (SW) Jacky Chiang , (SW) Kerwin Chen

Activities

(SW) Jacky Chiang

2021-04-19 11:25

manager   ~0006759

Hi Wooshin,

     Do you know what's SKB expected schedule? We'd like to know the feature priority because we have some features and issues that need to be implemented and resolved.

(SW) Jacky Chiang

2021-05-05 11:10

manager   ~0006994

Reminder sent to: (ALTech) Wooshin Kang

How about my question ?

(ALTech) Wooshin Kang

2021-05-05 12:56

developer   ~0006995

It should be finished to implement until beginning of June.

(ALTech) Wooshin Kang

2021-05-26 09:30

developer   ~0007232

hi Peter,

In step 2, you don't need to copy data for speaker. App will out mic data to speaker.

Thanks.

(ALTech) Wooshin Kang

2021-05-26 11:11

developer   ~0007244

Hi, Sita

I will explain again.

There are various karaoke APP and some app will use audio record class for making karaoke score.
So you have to make like below sequence.

1. check is it open "audio record class (MediaRecorder.AudioSource MIC type=VOICE_COMMUNICATION) from APP ?
2. If opened, send mic data to audio record class
3. If not opened, send mic data to SPEAKER or HDMI.

If you have any question, please inform me.

Thanks.

(ALTech) Wooshin Kang

2021-05-26 13:41

developer   ~0007250

This item will be controlled below link.
https://mantis.cnsbg.foxconn.com/vaas/view.php?id=1195#c7245

(SW) Peter Chen

2021-05-31 17:08

developer   ~0007307

Hi Wooshin,

As you mentioned in https://mantis.cnsbg.foxconn.com/vaas/view.php?id=1195#c7245, I tried with karaok APP (KaraokMicTest) for eagle.
But I find that there will have about 400ms delay between human voice to MIC and HDMI output. Is it still OK?

Thanks.

(ALTech) Wooshin Kang

2021-06-01 14:32

developer   ~0007319

Hi Peter,

I will check with SKB manager and share.

Thanks.

(ALTech) Wooshin Kang

2021-06-01 14:51

developer   ~0007320

Hi Peter,

Can you check delay after changing source code of APP ?

//mBufferSize = AudioRecord.getMinBufferSize(SAMPLE_RATE, AudioFormat.CHANNEL_IN_MONO, AUDIO_ENCODING);
mBufferSize = 256;

Thanks.

(SW) Peter Chen

2021-06-01 16:41

developer   ~0007322

Hi Wooshin,

I tried with BufferSize=256 and get the result below. It's still have 380ms delay time.

mBufferSize = AudioRecord.getMinBufferSize(SAMPLE_RATE, AudioFormat.CHANNEL_IN_MONO, AUDIO_ENCODING); --------> 420 ms
mBufferSize = 256; -----------> 380 ms

Thanks.

(ALTech) Wooshin Kang

2021-06-02 09:15

developer   ~0007324

Hi Peter,

Is it tested in smart3 ?
Can you check it also in swan ?

thanks.

(SW) Peter Chen

2021-06-02 09:46

developer   ~0007326

Hi Wooshin,

Yes, I tested in smart3.
For swan, please check with Sita in https://mantis.cnsbg.foxconn.com/vaas/view.php?id=1195#c7245.

Thanks.

(ALTech) Wooshin Kang

2021-06-02 10:54

developer   ~0007334

Last edited: 2021-06-02 10:55

 Hi Peter,

You can download karaoke test APP. Please check it.

https://drive.google.com/file/d/1xcQpk5SuSbgfd9lGv15wlL1K1ACkmFiF/view

Thanks.

(ALTech) Wooshin Kang

2021-06-02 16:41

developer   ~0007344

Hi Peter,

SKB wants to do follow 3 items.

1. share test image (smart3), they will do test for checking delay.
2. how to check delay, please explain detail steps
3. Can you check delay using attached karaoke app (https://drive.google.com/file/d/1xcQpk5SuSbgfd9lGv15wlL1K1ACkmFiF/view)

Thanks.

(SW) Peter Chen

2021-06-02 18:49

developer   ~0007345

Hi Wooshin,

1. share test image (smart3), they will do test for checking delay.
-----> I think you can use today's FW release v15.524.10 to do the test

2. how to check delay, please explain detail steps
-----> you can see the attachment testcase.pptx and detail steps below
           1. Open the karaoke test APP
           2. Use your phone to record the sound
           3. Start record (make sure to keep environment quiet)
           4. Make a sound to MIC (ex. knock MIC once)
           5. Stop record
           6. Open the recorded file by Audacity
           7. There might have two peak value (one is sound to MIC another is HDMI output)
           8. Check the time between two peak value (This is what we observed as delay time)

Thanks.
testcase.pptx (54,663 bytes)

(SW) Peter Chen

2021-06-04 12:12

developer   ~0007365

Hi Wooshin,

Some Info for your reference :
(Please be noted the info here is tested with no any change commit)

1. In the old karaoke app such as TJ, it will output the voice direct from HAL and no record. (delay 110-120ms)
2. In the Inopiatech test app, it will output the voice from APP not HAL and record data. (dleay 240-250ms)
3. In case2, open MIC by BTF service(output voice direct from HAL) at the same time and delay will become 380-400ms.

Please help to check what is the scenario of all the karaoke app that we may have in the future.

Thanks.

(SW) Peter Chen

2021-06-04 20:06

developer   ~0007370

Hi Wooshin,

Please check the question :

Q1. Is BTF SetMicOnOff(MIC_ON) called when karaoke app launches?
    1-1. app-release.apk(named skb) will not call SetMicOnOff(MIC_ON) when app launches.
    1-2. kumyoung app will call SetMicOnOff(MIC_ON) when app launches.
Q2. Do we need to do echo process when return MIC sound from AudioRecord()? Or will it be done in APP layer?
Q3. Is AudioRecord() called only when singing with song music?
Q4. If Q3 is yes, do we need to directly output MIC sound to output(HDMI/speaker) between song and song?
    For eaxmple, if user is operating on karaoke UI, do we need to directly output MIC sound to output(HDMI/speaker)?

Thanks.

(Attach is the recordflow)
recordflow.pptx (66,606 bytes)

(ALTech) Wooshin Kang

2021-06-07 14:00

developer   ~0007376

Hi Peter,

Actually, there is no exact information. I'll answer only as far as I know.

Q1. Is BTF SetMicOnOff(MIC_ON) called when karaoke app launches?
    1-1. app-release.apk(named skb) will not call SetMicOnOff(MIC_ON) when app launches. => Becaus
    1-2. kumyoung app will call SetMicOnOff(MIC_ON) when app launches.
==> It will be called when karaoke APP open.
Q2. Do we need to do echo process when return MIC sound from AudioRecord()? Or will it be done in APP layer?
==> Echo will be added by you, App will not control.

Q3. Is AudioRecord() called only when singing with song music?
==> You have to check it in app-release.apk

Q4. If Q3 is yes, do we need to directly output MIC sound to output(HDMI/speaker) between song and song?
    For eaxmple, if user is operating on karaoke UI, do we need to directly output MIC sound to output(HDMI/speaker)?
==> I think that you don't need to implement it.

Thanks.

(SW) Peter Chen

2021-06-08 13:41

developer   ~0007392

Last edited: 2021-06-08 13:41

 Hi Wooshin,

Thanks for your answer. Some questions need to check.

Q1. Is BTF SetMicOnOff(MIC_ON) called when karaoke app launches?
    1-1. app-release.apk(named skb) will not call SetMicOnOff(MIC_ON) when app launches.
          ===> We test with app-release.apk and it will not call BTF SetMicOnOff(MIC_ON) when karaoke APP open. Please be noted.
                    Attach log from kumyoung app and app-release.apk for reference.
    1-3. Will we have the karaoke app that we need to directly output MIC sound to output(HDMI/speaker) in the future?

Q4. If Q3 is yes, do we need to directly output MIC sound to output(HDMI/speaker) between song and song?
    For eaxmple, if user is operating on karaoke UI, do we need to directly output MIC sound to output(HDMI/speaker)?
==> I think that you don't need to implement it.
      >>>>> Please help to confirm that. Since it will affect our implement and schedule.

Thanks.

app-release.log (887,515 bytes)
tj.log (1,184,560 bytes)

(SW) Peter Chen

2021-06-16 15:05

developer   ~0007475

Hi Wooshin,

We found that SetMicOnOff(MIC_ON) is not called when app launches and not called when start to sing a song in the app-release.apk.
It's not match the flow you shared.
Attach the ref log from SWAN when open app and start to sing a song.

Please help to check with APP owner what is the detail process in karaoke app?
EX: 1.When to SetMicOnOff(MIC_ON/MIC_OFF)
      2.When to Start/Stop AudioRecord

Thanks.
newapp.log (975,450 bytes)
tj-2.log (1,227,364 bytes)

(SW) Peter Chen

2021-06-18 17:08

developer   ~0007505

Hi Wooshin,

Test image path : /release_by_fxn/smart3_test/Karaoke/k000b-524r10d_SD-20210618-BFX-AT100

Please be noted below.

1. The MIC setting function(DEVICE_SetMicOnOff) is not called in the new APP(app-release.apk, skb).
2. We remove audio_patch(MIC data direct output from HDMI) for this test image and it can work for APP(app-release.apk, skb) but not work for Kumyoung app.
3. Please verify if the MIC setting function(DEVICE_SetMicOnOff) will be called in new APP or not with APP vendor. It will affect our schedule for feature implementation.
4. The voice from audioTrack of new APP sounds very quiet, so we think the volume of voice maybe need to be amplified in APP.
5. Other audio feature(including audio routing, NUGU, youtube, and BTV) have been not fully tested yet. We cannot ensure these feature is working normally.

Thanks.

(ALTech) Wooshin Kang

2021-06-21 18:25

developer   ~0007509

Hi Peter,

Thanks for image.
I will test it and share the result.

(ALTech) Wooshin Kang

2021-06-25 10:00

developer   ~0007554

Hi Peter,

SKB decide to go original way due delay.
< Original Idea >
1. Please keep sending data to amp driver to output from speaker.
2. Copy data for speaker from step1 and send it to audio record class.
3. Android Audio Record Class get data using "MediaRecorder.AudioSource MIC type=VOICE_COMMUNICATION"
4. Some Apk will use it. (example, Karaoke app uses it to get score)

I am sorry for confusing.
Thanks.

(ALTech) Wooshin Kang

2021-06-28 19:03

developer   ~0007571

Hi Peter,

You can download new test app from below link.
https://mantis.cnsbg.foxconn.com/vaas/view.php?id=1195
Thanks.

(SW) Peter Chen

2021-07-14 09:29

developer   ~0007670

Last edited: 2021-07-14 09:32

 Hi Wooshin,

Test image has been uploaded , file path: /release_by_fxn/smart3_test/Karaoke/0714/k000b-524r10l_SD-20210714-BFX-AT100

Some notes for this test image:
1. Enable MIC in default for testing.
2. The voice will output from HAL and APP in app-release.apk so you will hear your voice twice.
3. In the app-release-5th.apk, the score is not work.
4. The delay time is about 150 ms now. We will check how to reduce the delay time.
5. Other audio feature have not been fully tested yet.
6. Please confirm with APP provider vendor if MIC setting function(DEVICE_SetMicOnOff) will be called. We still need some time to change code.

Thanks.

(SW) Brent Choi

2021-07-19 13:46

no access  

app-release_7th.apk (297,194,402 bytes)

(SW) Brent Choi

2021-08-10 17:09

no access  

app-release_8th.apk (300,174,988 bytes)

(SW) Kerwin Chen

2021-12-23 16:37

developer   ~0009058

closed, finish implementation

Issue History

Date Modified Username Field Change
2021-04-13 08:53 (ALTech) Wooshin Kang New Issue
2021-04-13 08:53 (ALTech) Wooshin Kang Status new => assigned
2021-04-13 08:53 (ALTech) Wooshin Kang Assigned To => (SW) Jacky Chiang
2021-04-19 11:25 (SW) Jacky Chiang Note Added: 0006759
2021-04-26 09:35 (SW) Jacky Chiang Assigned To (SW) Jacky Chiang => (ALTech) Wooshin Kang
2021-04-26 09:36 (SW) Jacky Chiang Issue Monitored: (ALTech) Younkwang Jung
2021-05-05 11:10 (SW) Jacky Chiang Note Added: 0006994
2021-05-05 12:56 (ALTech) Wooshin Kang Note Added: 0006995
2021-05-14 18:35 (ALTech) SY Yoon Assigned To (ALTech) Wooshin Kang => (SW) Jacky Chiang
2021-05-14 18:36 (ALTech) SY Yoon Due Date 2021-04-30 09:52 => 2021-06-03 09:52
2021-05-17 09:45 (SW) Jacky Chiang Assigned To (SW) Jacky Chiang => (SW) Peter Chen
2021-05-17 09:46 (SW) Jacky Chiang Issue Monitored: (SW) Jacky Chiang
2021-05-17 09:46 (SW) Jacky Chiang Issue Monitored: (SW) Kerwin Chen
2021-05-26 09:30 (ALTech) Wooshin Kang Note Added: 0007232
2021-05-26 11:11 (ALTech) Wooshin Kang Note Added: 0007244
2021-05-26 13:41 (ALTech) Wooshin Kang Note Added: 0007250
2021-05-31 17:08 (SW) Peter Chen Note Added: 0007307
2021-06-01 14:32 (ALTech) Wooshin Kang Note Added: 0007319
2021-06-01 14:51 (ALTech) Wooshin Kang Note Added: 0007320
2021-06-01 16:41 (SW) Peter Chen Note Added: 0007322
2021-06-02 09:15 (ALTech) Wooshin Kang Note Added: 0007324
2021-06-02 09:46 (SW) Peter Chen Note Added: 0007326
2021-06-02 10:54 (ALTech) Wooshin Kang Note Added: 0007334
2021-06-02 10:55 (ALTech) Wooshin Kang Note Edited: 0007334
2021-06-02 16:41 (ALTech) Wooshin Kang Note Added: 0007344
2021-06-02 18:49 (SW) Peter Chen File Added: testcase.pptx
2021-06-02 18:49 (SW) Peter Chen Note Added: 0007345
2021-06-04 12:12 (SW) Peter Chen Note Added: 0007365
2021-06-04 20:06 (SW) Peter Chen File Added: recordflow.pptx
2021-06-04 20:06 (SW) Peter Chen Note Added: 0007370
2021-06-07 14:00 (ALTech) Wooshin Kang Note Added: 0007376
2021-06-08 13:41 (SW) Peter Chen File Added: app-release.log
2021-06-08 13:41 (SW) Peter Chen File Added: tj.log
2021-06-08 13:41 (SW) Peter Chen Note Added: 0007392
2021-06-08 13:41 (SW) Peter Chen Note Edited: 0007392
2021-06-16 15:05 (SW) Peter Chen File Added: newapp.log
2021-06-16 15:05 (SW) Peter Chen File Added: tj-2.log
2021-06-16 15:05 (SW) Peter Chen Note Added: 0007475
2021-06-18 17:08 (SW) Peter Chen Note Added: 0007505
2021-06-21 18:25 (ALTech) Wooshin Kang Note Added: 0007509
2021-06-25 10:00 (ALTech) Wooshin Kang Note Added: 0007554
2021-06-28 19:03 (ALTech) Wooshin Kang Note Added: 0007571
2021-07-14 09:29 (SW) Peter Chen Note Added: 0007670
2021-07-14 09:32 (SW) Peter Chen Note Edited: 0007670
2021-07-19 13:46 (SW) Brent Choi File Added: app-release_7th.apk
2021-08-10 17:09 (SW) Brent Choi File Added: app-release_8th.apk
2021-12-23 16:37 (SW) Kerwin Chen Status assigned => closed
2021-12-23 16:37 (SW) Kerwin Chen Resolution open => fixed
2021-12-23 16:37 (SW) Kerwin Chen Note Added: 0009058